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Abstract 


This report focuses on the extent to which students who are academically far off track in 
fourth or eighth grade in Kentucky catch up by eighth or eleventh grade. We studied three recent 
cohorts of Kentucky students whose eighth-grade ACT Explore® scores were more than one 
standard deviation below the ACT Explore benchmark scores associated with being on track. We 
found that 5% or fewer of the students who were far off track in eighth grade attained the ACT 
College Readiness Benchmarks® by the spring of eleventh grade. We did a similar analysis for 
two cohorts of students beginning in fourth grade, using scores on the Kentucky Core Content 
Tests in reading, mathematics, and science in grade four and ACT Explore scores in the same 
subjects in grade eight, and found that catching-up rates ranged from 7% in mathematics to 12% 
in science. We also found that students from at-risk groups — those who are low-income, African 
American, Hispanic, English language learners, or in special education — have lower catching up 
rates than their more advantaged peers. These results are of special concern because a large 
percentage of students from those groups are far off track in fourth and eighth grade. 

These findings should underscore the importance of policies and practices that focus on 
getting students off to a good start in the early grades. These practices are especially important 
for disadvantaged students. Ongoing research should identify practices that help to accomplish 
this goal, and state and local policy should support efforts to disseminate and implement those 
practices. 
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Introduction 

In recent years, educators and policymakers have set a goal that students graduate from 
high school ready for college and careers. However, as a nation we fall short of achieving this 
goal, particularly for disadvantaged students. In states where the highest percentages of students 
took the ACT Assessment® in 2012, for example, 45% of students in the two lowest family 
income categories" met ACT’s national College Readiness Benchmarks in English, 21% in 
mathematics, 24% in reading, and 18% in science. In Kentucky, of 15,163 students from these 
two family income categories who took the ACT, 44% met the ACT benchmark in English, 17% 
in mathematics, 24% in reading, and 16% in science. 

A substantial body of research supports the idea that the path to college and career 
readiness begins in early childhood. Gaps in vocabulary development begin in very early 
childhood (Hart & Risley, 1995), and students entering kindergarten from disadvantaged 
backgrounds tend to lag behind their more advantaged peers in vocabulary and overall oral 
language development (Farkas & Beron, 2004; Dunham, Farkas, Hammer, Tomblin, & Catts, 
2007) as well as in early reading and mathematics skills and background knowledge (West, 
Denton, & Germino-Hausken, 2000). In turn, early reading and mathematics skills and 
background knowledge predict student success in the later grades (Duncan, Claessens, Huston, 
Pagani, Engel, Sexton, Dowsett, Magnuson, Klebanov, Feinstein, Brooks-Gunn, Duckworth, & 


1 This study uses data maintained by the Kentucky Department of Education and is published with its permission. 

2 These two income categories together consist of students with a self-reported family income of less than $36,000 a 
year. 

3 These statistics are based on the updated benchmarks of 22 in reading and 23 in science calculated in Allen (2013). 
The states with the highest percentages of students taking the ACT were Colorado, Illinois, Kentucky, Michigan, 
North Dakota, Tennessee, and Wyoming. The data file contained the most recent ACT scores of students who were 
twelfth graders in 2012; ACT scores for students who did not take the ACT in twelfth grade came from earlier 
grades and years. 
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Japel, 2007; Claessens & Engel, 2013; Grissmer, Grimm, Aiyer, Murrah, & Steele, 2010; Geary, 
2011). With these findings in mind, recent ACT reports have focused on the importance of 
getting students off to a good start in preschool and the early elementary grades (Sawyer, 2008; 
Sawyer & Gibson, 2012; ACT, 2012a; Dougherty, 2013). 

Learning gaps that emerge early are likely to widen over time because of “Matthew 
effects,” whereby those who start out ahead are at a relative advantage in acquiring new 
knowledge (Stanovich, 1986). These effects can occur because students who already know about 
a topic often find it easier to leam new information on the same topic (Willingham, 2006), and 
because prior exposure to knowledge can motivate students to learn more (Durik & Matarazzo, 
2009; Maltese & Tai, 2010). In addition, in order to catch up, students who are academically off 
track must grow faster than students ahead of them. The lagging students must do double duty, 
catching up on content that they missed earlier while mastering newly taught curriculum. 
Students who are already on track do not carry this extra burden. 

This report follows up on the analysis in a recent ACT research report (Dougherty & 
Fleming, 2012). We used the recently updated ACT College Readiness Benchmarks (Allen, 
2013) as a measure of high school students’ academic preparation for two- and four-year 
colleges and other postsecondary training programs leading to skilled careers (ACT, 2006). 
These benchmarks identify the ACT scores associated with a 50% probability of earning a B or 
higher, or a 75% chance of earning a C or higher, in entry-level college courses corresponding to 
the ACT subject tested (Allen & Sconing, 2005). We used the corresponding ACT Explore 
College Readiness Benchmarks as indicators of whether eighth grade students are on track to 
meet the ACT benchmarks. The report focused on students who start out far off track — scoring 
more than one standard deviation below the ACT Explore Benchmark in eighth grade in a given 
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subject, or more than one standard deviation below the fourth-grade state test score associated 
with a 50% probability of reaching the ACT Explore Benchmark in eighth grade. 

The next section discusses the methodology of our study. The two following sections 
look at results for students catching up in Grades 8-11 and Grades 4-8. Subsections of the report 
examine the percentage of far-off-track students from different demographic groups who reached 
the benchmarks four years later and how far short of the benchmarks the other students fell. 
Finally, the conclusion discusses implications of our findings for how educators and 
policymakers should think about intervention and accountability requirements. 

Methodology 

Students in the Analysis 

Grades 8-11. For the analysis of students catching up in high school, we used data from 
three cohorts of Kentucky students who took the ACT Explore test in Grade 8 in the 2006-07, 
2007-08, or 2008-09 school years and the ACT in Grade 1 1 three and a half years later (Table 1). 
This analysis was possible in Kentucky because state education officials provided student-level 
enrollment, state test, ACT Explore, and ACT data from the state’s longitudinal data system that 
could be matched across datasets and years using the state student ID. This made it possible to 
link the ACT Explore and ACT results to state enrollment and academic achievement test data, 
and to disaggregate ACT Explore and ACT results based on state -provided student demographic 


information. 
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Table 1 


Kentucky Grades 8-11 Student Cohorts 


Student cohort 

School year 
taking ACT 
Explore 

School year 
taking 
the ACT 

Grade taking 
the ACT 

Number of 
students* 

2007-2010 

2006-07 

2009-10 

11 

30,575 

2008-2011 

2007-08 

2010-11 

11 

30,755 

2009-2012 

2008-09 

2011-12 

11 

34,012 

Total (3 cohorts) 



95,342 


* 1 % of students in these cohorts with incomplete demographic records were dropped from the 
analysis. 


The data used for the longitudinal study contained about 68% of the population of eighth 
grade tested students in Kentucky. 4 This percentage is relatively high since Kentucky administers 
ACT Explore and the ACT statewide. As expected, the students in the longitudinal cohorts in the 
study — representing students who stayed in school, followed a normal grade progression, 
remained in the state, and took both ACT Explore and the ACT — had lower percentages of 
students from typically at-risk groups than did the general population of eighth grade students. 
For example, students in the study were less likely to be low-income, African American, or in 
special education (Table 2). 5 This means that any difficulty that students in the study had 
catching up would likely be amplified in the general student population, thus lending greater 
weight to the findings of this study about the difficulties experienced by far off track students. 


4 From Table 2, the 95,342 Kentucky cohort students were about 68% of the total eighth grade tested population of 
140,827 students from the cohorts’ eighth grade years. The database did not include twelfth grade ACT scores from 
Kentucky students retaking the test as high school seniors. 

5 In Table 2, the demographic characteristics of each student was taken from the eighth grade data, so that 
comparison between longitudinal and eighth grade snapshot cohorts use the same information on each student. 
Moving from left to right, students in each column of Table 2 are a subset of those in the previous column. 
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Table 2 

Demographics of Kentucky 8 th Grade Students from 2006-07, 2007-08, and 2008-09 School 
Years 



Total 8th grade 
tested student 
population* 

8th graders taking 
ACT Explore 

8th grade ACT 
Explore takers in 
longitudinal 
cohorts 

Number of students* 

140,827 

128,511 

95,342 

Percent low-income 

50 

49 

43 

Percent African American 

10 

10 

9 

Percent Hispanic 

2 

2 

2 

Percent English language 
learners 

1 

1 

1 

Percent special education 

12 

11 

9 


* Students from the Kentucky Core Competency Test (KCCT) databases for the three school years. 1% 
of students with incomplete demographic records were dropped from the analysis. 


Grades 4-8. For students in upper elementary and middle school, we used data from two 
cohorts of Kentucky students who took the Kentucky Core Competency Test (KCCT) in reading, 
mathematics, and/or science in Grade 4 in the 2006-07 or 2007-08 school years and ACT 
Explore in Grade 8 four years later (Table 3). The students in the longitudinal cohorts — who 
followed a nonnal grade progression, stayed in the state, and took the ACT Explore test in eighth 
grade — were similar demographically to the overall population of all fourth grade tested students 
(Table 4). Differences in the percentages of African American, Hispanic, English language 
learner, and special education students between the longitudinal cohort and the general student 
population were small, showing up only in the first decimal place (not shown in Table 4). 
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Table 3 

Kentucky Grades 4-8 Student Cohorts 


School year 
taking 4th 

Student cohort grade test 

School year 
taking ACT 
Explore 

Grade taking 
ACT Explore 

Number of 
students* 

2007-2011 2006-07 

2010-11 

8 

35,205 

2008-2012 2007-08 

2011-12 

8 

39,368 

Total (2 cohorts) 



74,573 

* 1.9% of students in these cohorts with incomplete demographic records were 
analysis. 

dropped from the 

Table 4 




Demographics of Kentucky 4 th Grade Students from 2006-07 and 2007-08 School Years 


Total 4th grade 
enrollment* 

Tested 4th 
graders 

Tested 4th 
graders in 
longitudinal 
cohorts 

Number of students 

n/a 

87,169 

74,573 

Percent low-income 

n/a 

52 

51 

Percent African American 

n/a 

10 

10 

Percent Hispanic 

n/a 

2 

2 

Percent English language 
learners 

n/a 

2 

2 

Percent special education 

n/a 

13 

13 


* The enrollment data provided by Kentucky begins with the 2008-09 school year. The second 
column of this table, KCCT-tested students, is analogous to the first column of Table 2. 









7 


Disaggregation of Students into Demographic Groups 

Grades 8-11 and 4-8. In addition to looking at all tested students, we disaggregated the 
students in the longitudinal cohorts and the ACT Explore tested population as a whole into 19 
additional demographic subgroups, making 20 groups altogether. The first set of eight groups 
consists of: 

1 . All students 

2. Low-income students 

3. Non-low-income students 

4. African American students 

5. Hispanic students 

6. Other students 6 

7. English language learners 

8. Special education students 7 

An additional set of 12 mutually exclusive groups disaggregates students by income, ethnicity, 
and gender: 

9. Low-income African American males 

10. Low-income African American females 

1 1 . Low-income Hispanic males 

12. Low-income Hispanic females 

13. Other low-income males 


6 The “Other” group consists of individuals who are neither African American nor Hispanic. In Kentucky, the great 
majority of those students are White. The three ethnic groups (4-6) are mutually exclusive, as are the two income 
groups (2 and 3). 

7 In addition, we looked at results for the following additional four groups not reported on in this paper: Hispanic 
English language learners, non-Hispanic English language learners, non-English language learners, and students not 
in special education. 
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14. Other low-income females 

15. Non-low-income African American males 

16. Non-low-income African American females 

17. Non-low-income Hispanic males 

18. Non-low-income Hispanic females 

19. Other non-low-income males 

20. Other non-low-income females 

To keep the number of statistics in the paper to a manageable size, this report focuses on the first 
eight groups. However, information on students disaggregated by income, ethnicity, and gender 
(groups 9 through 20) is available in Appendix B. 

Division of Students into Academic Preparation Groups 

Grades 8-11. We classified eighth grade students into three academic preparation groups 
in each of four subject areas (English, mathematics, reading, and science) based on their 
performance on ACT Explore in these areas: 

• “On-Track” students met the College Readiness Benchmark score on ACT Explore 
(Table 5) in the subject. 8 

• “Off-Track” students missed the Benchmark by one standard deviation or less. 


s Readers should note that reaching the College Readiness Benchmark on ACT Explore in the eighth grade does not 
imply that the student is college-ready in eighth grade, only that he or she is on track to being college ready on the 
ACT by eleventh or twelfth grade. 
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• “Far-Off-Track” students scored more than a full standard deviation below the 
Benchmark. 9 


Table 5 

Student Academic Preparation Levels on ACT Explore (Grade 8) 


Academic 
preparation level 

English 

Mathematics 

Reading 

Science 

On Track 

Met or exceeded the 
College Readiness 
Benchmark 

13 or above 

17 or above 

16 or above 

1 8 or above 

Off Track 

No more than one 
standard deviation 
below the 
Benchmark 

9- 12 

14- 16 

13 - 15 

15 - 17 

Far Off Track 

More than one 
standard deviation 
below the 
Benchmark 

8 or below 

13 or below 

12 or below 

14 or below 


For example, a score of 16 or better in ACT Explore Reading indicated that a student was 
On Track; Off Track students scored from 13 to 15, while students scoring 12 or below were 
classified as Far Off Track. Similarly, Table 6 shows the ACT scores indicating that a student is 
On Track (meeting the ACT College Readiness Benchmark), Off Track, or Far Off Track. These 
benchmarks, first set in 2005, were updated in 2013 based on more recent data linking students’ 
ACT scores to their grades in credit-bearing first-year college courses (Allen, 2013; Allen & 


9 Standard deviations were chosen as the yardstick because they provide a common metric across different grades 
and tests. A one-standard deviation difference in scores is quite large: in reading and mathematics, it is roughly the 
difference between scoring at the Basic and Proficient levels on the National Assessment of Educational Progress 
(NAEP), or between the 16 th and the 50 th percentiles on a norm-referenced standardized test. The size of a standard 
deviation on ACT Explore (based on national data) was 4.2 points in English, 3.5 in mathematics, 3.9 in reading, 
and 3.3 in science. Standard deviations on the ACT were 6.4 points in English, 5.3 in mathematics, 6.2 in reading, 
and 5.1 in science. 
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Sconing, 2005). In the update, the English and mathematics benchmarks remained the same, 
while the ACT benchmark in reading changed from 21 to 22 and in science from 24 to 23. 
Similarly, the eighth grade ACT Explore benchmark changed from 15 to 16 in reading and 20 to 
18 in science. We used these updated benchmarks for all years of data in this report. 

Table 6 

Student Academic Preparation Levels on the ACT 


Academic 
preparation level 

English 

Mathematics 

Reading 

Science 

On Track 

Met or exceeded the 
College Readiness 
Benchmark 

1 8 or above 

22 or above 

22 or above 

23 or above 

Off Track 

No more than one 
standard deviation 
below the 
Benchmark 

12 - 17 

17-21 

16-21 

18-22 

Far Off Track 

More than one 
standard deviation 
below the 
Benchmark 

1 1 or below 

16 or below 

15 or below 

17 or below 


In ACT Explore English, few students scored at the Far Off Track level of 8 or below. 

Thus, we focused our analysis of Far Off Track students in the other three subjects. 

Grades 4-8. Similarly, we classified fourth grade students in the two Kentucky cohorts as 

On Track, Off Track, and Far Off Track based on their scores on the Kentucky Core Competency 

Tests. 10 This classification was based on a direct link between students’ fourth grade KCCT 

scores in reading, mathematics, and science and their eighth grade ACT Explore scores in the 

10 In the 2011-12 school year, the KCCT tests in grades 3-8 were replaced by the Kentucky Performance Rating for 
Educational Progress (K-PREP) tests. The first group of fourth graders taking K-PREP will be eighth graders in the 
2015-16 school year. 
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same subjects. This was done by using logistic regression to identify the fourth grade KCCT 
score in each subject associated with a 50% or better probability of meeting or exceeding the 
eighth grade benchmark on ACT Explore in the corresponding subject. 11 Data from both cohorts 
was combined for this analysis. This led to the identification of the fourth grade academic 
preparation levels shown in Table 7. 12 
Table 7 

Student Academic Preparation Levels on the Grade 4 Kentucky State Test 


Academic 
preparation level 

Reading 

Mathematics 

Science 

On Track 

Met or exceeded the 

468 

466 

465 

College and Career 
Readiness Target 

or above 

or above 

or above 

Off Track 

No more than one 
standard deviation 
below the Target 

449 - 467 

445 - 465 

447 - 464 

Far Off Track 

More than one standard 

448 

444 

446 

deviation below the 
Target 

or below 

or below 

or below 


11 Students were classified into two categories based on whether they did or did not meet the ACT Explore 
benchmark in a subject, and a logistic regression model such as the one described in Allen (2013) was used to assess 
the probability of meeting the ACT Explore benchmark in the subject as a function of the student’s fourth grade 
KCCT score in the same subject. 

12 These preparation levels are slightly different from those identified using different years’ data or a different 
linking methodology, e.g., the methodology used in ACT (2012b). Most of the difference between the On Track 
reading target shown in Table 7 and on page 3 of ACT (2012b) is due to the use of the updated ACT Explore 
reading benchmark. Once that is taken into account, the On Track targets differ by one KCCT scale score point 
(about .05 standard deviation) in reading and two KCCT scale score points (about .09 standard deviation) in 
mathematics. This small difference is due to the use of different years of data (fourth grade in the 2009-10 school 
year in the earlier publication, versus fourth grade in the 2006-07 and 2007-08 school years in this paper) and a 
different linking methodology (direct links from fourth to eighth grade in this paper, versus direct links from seventh 
to eighth grade and statistical moderation from seventh grade down to fourth grade in the earlier publication). 
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Classification of Far-Off-Track Students Based on their Amount of Catching Up 

Grades 8-11. For each Far Off Track student in a given subject, we calculated the 
difference between the student’s ACT Explore score and the ACT Explore College Readiness 
Benchmark in that subject. For example, consider a hypothetical student with a score of 10 on 
the ACT Explore reading test. This student has a scale score gap of -6 relative to the ACT 
Explore Benchmark score of 16. (The gap is negative to emphasize that the student falls short 
of the Benchmark.) If the same student scores 16 on the ACT, the student’s scale score gap is -6 
relative to the ACT Reading Benchmark of 22, the same as the student’s scale score gap on ACT 
Explore. An ACT reading score of 19, on the other hand, would constitute a gap of -3, and the 
student would have closed half of the ACT Explore score gap. A student scoring at or above the 
Benchmark on the ACT would have closed all of the gap. 

Score gaps can also be measured in standard deviation units. In that case, we refer to 
them as z-score gaps. The definitions of Off Track and Far Off Track students in Tables 5-7 are 
based on z-score gaps. For example, our hypothetical student with an ACT Explore reading score 
of 10 has a z-score gap of about -1.54 (i.e., -6/3.9, where 3.9 is the standard deviation of ACT 
Explore reading scores). This places the student in the Far Off Track group, as the student scores 
more than one standard deviation below the Benchmark. If the same student scores 16 on the 
ACT, the scale score gap is unchanged but the z-score gap narrows to -.97 (= -6/6.2, where 6.2 
is the standard deviation of ACT reading scores), and the student is counted in the Off Track 
group. Z-score gaps adjust for the wider dispersion of student scores on the ACT than on ACT 
Explore. Analogously, a 15-pound weight gap is a larger share of typical weight differences 

13 “Scale scores” are the familiar ACT Explore and ACT scores reported on a scale from 1 to 25 and 1 to 36, 
respectively. They are distinguished from “raw scores” which represent the percentage of test items correct, and 
other forms of score reporting such as stanines, percentile ranks, and grade equivalents that are used on norm- 
referenced tests. 
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among 5-year-olds than it is among 30-year-olds, so a five-year-old who is 15 pounds 
overweight might be considered to be “more overweight” than a 30-year-old who is 15 pounds 
overweight. This may be small consolation to the 30-year-old who must still make the effort to 
lose 15 pounds. So both score gaps and z-score gaps are useful measures of how far students fall 
short of On Track benchmarks. 

Using score gaps as the metric, we classified students who scored Far Off Track on a 
given subject on ACT Explore into four scale score growth categories based on how much they 
closed their scale score gaps in the same subject when they took the ACT (Table 8). Likewise, 
we divided those students into four z-score growth categories based on how close they came to 
reaching the ACT Benchmarks (Table 9). 

Table 8 

ACT Explore-ACT Scale Score Growth Categories for Far Off Track Students 


Category 1 

“Reached Benchmark”: the student closed the entire ACT Explore 
scale score gap by scoring at or above the ACT College Readiness 
Benchmark. 

Category 2 

“Closed half or more of gap”: the student’s ACT Explore scale 
score gap narrowed by half or more on the ACT. 14 

Category 3 

“Closed gap by less than half’: the student’s ACT Explore scale 
score gap narrowed on the ACT, but by less than half. 

Category 4 

“No gap closing”: the student’s ACT Explore scale score gap stayed 
the same or widened on the ACT. 


14 This is a less stringent criterion than proposed in ACT (2009), which suggested that Off-Track students be 
expected to close half the point gap between ACT Explore and PLAN and half of the gap again between PLAN and 
the ACT, or three-quarters of the gap altogether between ACT Explore and the ACT. 
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Table 9 

ACT Explore-ACT z-Score Growth Categories for Far Off Track Students Based on the Change 
in the Student ’s Academic Performance Level 


Category 1 

“Reached Benchmark”: the student moved from Far Off Track on 
ACT Explore to scoring at or above the ACT College Readiness 
Benchmark. 

Category 2 

“Reached top half of Off Track level”: the student moved from Far 
Off Track to scoring in the top half of the Off Track performance 
level on the ACT. 15 

Category 3 

“Reached bottom half of Off Track level”: the student moved from 
Far Off Track to scoring in the bottom half of the Off Track 
perfonnance level on the ACT. 

Category 4 

“Stayed Far Off Track”: the student scored in the Far Off Track 
category on the ACT. 


To see how students are placed in these categories, consider our far-off-track eighth- 
grade student with an ACT Explore reading score of 10, representing a scale score gap of -6 and 
a z-score gap of -1.54 relative to the Explore reading benchmark of 16. If the student later scores 
at or above the College Readiness Benchmark of 22 on the ACT reading test, that student attains 
Category 1 on both growth metrics. To reach Category 2 on scale score growth, the student must 
score 19-21 on the ACT, reducing the scale score gap relative to the Benchmark to 3 points or 
less. To reach Category 2 on z-score growth, the student must score no more than one-half 
standard deviation below 22, also a score of 19-21. (A half standard deviation in ACT reading is 
6.2/2 = 3.1 points.) For Category 3 on scale score growth, the student must score 17 or 18 on the 
ACT; a score of 16 or below would fail to narrow the 6-point gap relative to the Benchmark, thus 
leaving the student in Category 4. In comparison, an ACT score of 16-18 places the student in 


15 The ACT scores required to reach this category are 15 in English, 20 in mathematics, 19 in reading, and 21 in 


science. 
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Category 3 on z-score growth, since the borderline between Category 3 and 4 — one standard 
deviation below the Benchmark — falls between an ACT score of 15 and 16. 16 

Scale score growth measures between two tests depend on the tests having a common 
vertical scale, as is the case for ACT Explore and the ACT. Z-score growth measures can be used 
even if no such scale exists, as long as the subject matter of the two tests is similar enough for 
the concept of “growth” to be meaningful. 

Because the scores of any predefined group of students contains an error component that 
is positive on average for students chosen from near the top of the score distribution and negative 
on average for students chosen from near the bottom, all averages of groups of students chosen 
based on their prior performance tend to move back toward the average of all students. Using a 
sports analogy, a group of baseball players chosen for the highest batting averages in the first six 
weeks of the season will probably bat at a lower average for the rest of the season, even if they 
continue to bat well above the average for all players (Campbell & Kenny, 1999). This 
regression effect tends to reduce the expected future growth of On Track students and increase 
the expected future growth of Far Off Track students. 17 Possibly offsetting this regression effect 
are Matthew effects, which give an advantage to the students with better prior academic 
preparation. 18 


16 Note that the z-score categories only depend on the student’s ending point, whereas the scale score categories also 
depend on how far behind the Far-Off-Track student starts. 

17 In eighth grade, the cut scores for being On Track (shown in Table 5) were slightly below the Kentucky ACT 
Explore score mean of 13.9 in English but above the Kentucky means of 14.6 in mathematics, 14.0 in reading, and 
16.0 in science. Thus any On Track score was above the state average in three of the four subjects. 

18 Faster growth of groups of previously higher performing students than of groups of lower performing students is 
highly suggestive of Matthew effects. In fact, because of regression to the mean, the same growth by groups of 
previously higher performing students could be suggestive of these effects. On the other hand, the fact that 
individual scores diverge over time is not in itself proof of Matthew effects, as individual scores will spread out over 
time even if current period score growth is uncorrelated with growth or performance levels in prior periods - think 
of the spreading out of an ink blot even if all movement of ink molecules is random and unrelated to prior position 
or movement. Note also that growth comparisons of student groups that begin at different levels depend on the 
assumption that the score scale has equal-interval properties - that is, growth from 10 to 15 has the same meaning as 
growth from 20 to 25. 
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Scale scores but not z-scores can be used to look for Matthew effects, as z-scores “adjust 
out” the increased spread of scores over time, including any divergence of groups of students that 
might be attributable to Matthew effects. Z-score measures do not, on the other hand, remove 
regression effects. Therefore any z-score measure will always show the top students declining 
and the bottom students improving, even if their scale scores diverge. Thus, it is not sufficient 
just to see if Far Off Track students’ z-scores improve, but whether they improve enough to 
bring them close to the Benchmarks. That is the reason for emphasizing the student’s ending 
point when looking at z-scores. 

The academic preparation level measures (Far Off Track, Off Track, and On Track) and 
growth category measures used in this report are subject-specific. A student might be Far Off 
Track in ACT Explore reading and/or achieve little growth in that subject between ACT Explore 
and the ACT, but perform very well in ACT Explore mathematics and/or achieve strong 
mathematics growth between ACT Explore and the ACT. 

Grades 4-8. We used students’ Kentucky Core Competency Test scores in reading, 
mathematics, and science in Grade 4 and the ACT Explore scores in reading, mathematics, and 
science in Grade 8 as our endpoints for student growth. As the KCCT and ACT Explore tests 
were not scored on a common vertical scale, no scale score growth measure exists between the 
two sets of tests. 19 However, academic performance levels and growth measure can be calculated 
based on the number of standard deviations that students scored below the On Track measure on 
each test. Thus, fourth grade Far Off Track students were classified into the four z-score growth 
categories shown in Table 9 based on the performance level they reached on ACT Explore in 
eighth grade. 

19 For example, while the ACT Explore is scored on a scale from 1 to 25 in each subject, the 2008 fourth grade 
KCCT scale scores ranged from 400 to 480 in each subject. 
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Because z-score growth measures were available at both levels (grades 4-8 and 8-11), we 
focus on those measures in the main body of this report. Appendix A contains a discussion of 
scale score growth by Kentucky students in grades 8-11. 

Results 

Closing Academic Preparation Gaps in High School 
What Percentage of Students Were Far Off Track in 8 th Grade? 

Table 10 shows the percentage of ACT Explore eighth grade test-takers in Kentucky who 
were Far Off Track in the 2006-07, 2007-08, and 2008-09 school years, the starting years for the 
students in the three cohorts in this study. Between 29 and 38 percent of students in the overall 
population were Far Off Track in 2006-07, 2007-08, and 2008-09 (top row of Table 10), 
depending on the subject tested. These percentages were higher for students in at-risk student 
groups, as can be seen from the remaining rows. For example, roughly 44, 49, and 39 percent of 
low-income students were Far Off Track in mathematics, reading, and science, respectively. 
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Table 10 

Percentage of Kentucky ACT Explore Tested 8 th Grade Students Who Were Far Off Track from 
2006-07, 2007-08, and 2008-09 School Years 


Percentage of 8th graders who were Far 
Off Track 


Group Number of 


# 

Group 

students 

Mathematics 

Reading 

Science 

1 

All students 

128,511 

32 

38 

29 

2 

Low-income 

62,651 

44 

49 

39 

3 

Non-low-income 

65,860 

21 

27 

20 

4 

African American 

12,291 

52 

57 

43 

5 

Hispanic 

2,360 

42 

48 

36 

6 

Other 20 

113,860 

30 

36 

28 

7 

English language learners 

966 

61 

71 

51 

8 

Special education 

14,437 

70 

67 

61 


As Table 1 1 shows, the percentages of Far Off Track students were lower for students in 
the longitudinal cohorts — who stayed in school, made normal progress through the grades, 
remained in the state, and took the ACT college readiness assessment in grade 1 1 . Not only were 
the students in these cohorts less likely to be from at-risk groups (Table 2), but the longitudinal 


20 As mentioned in an earlier footnote, the “Other” group consists of students who are neither African American nor 
Hispanic. Thus, the sum of the number of students in the African American, Hispanic, and Other groups equals the 
total number of students in the top row. Likewise, the number of low-income and non-low-income students adds up 
to the number in the top row. 
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cohort students from at-risk groups were less likely than others in their demographic groups to be 
Far Off Track. 21 
Table 1 1 

Percentage of Kentucky ACT Explore Tested 8 th Grade Students Who Were Far Off Track from 
Students in Longitudinal Cohorts, 2006-07, 2007-08, and 2008-09 School Years 


Percentage of 8th graders who were Far 
Off Track 


Group Number of 


# 

Group 

students 

Mathematics 

Reading 

Science 

1 

All students 

95,342 

26 

33 

24 

2 

Low-income 

41,003 

37 

43 

32 

3 

Non-low-income 

54,339 

18 

25 

17 

4 

African American 

8,160 

46 

51 

37 

5 

Hispanic 

1,457 

35 

41 

29 

6 

Other 

85,725 

24 

31 

22 

7 

English language learners 

532 

55 

65 

45 

8 

Special education 

8,969 

66 

62 

56 


21 For example, the percentages of Far Off Track students among the eighth grade low-income students not in the 
longitudinal cohorts were 57%, 61%, and 51% in mathematics, reading, and science, respectively. 
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What Percentage of Far Off Track 8 th Graders Reached College Readiness Benchmarks by 
11 th Grade? 

Based on an analysis of the Kentucky longitudinal cohorts, few Far Off Track eighth 
grade students reached the ACT College Readiness Benchmarks by the spring of eleventh grade. 
Only about 1% of Far Off Track eighth graders met the ACT Benchmark in mathematics, 5% in 
reading, and 3% in science (Figure 1). “ Success rates for Off Track students were higher: 13% 
in mathematics, 25% in reading, and 14% in science. By contrast, the majority of On Track 
eighth graders met the ACT Benchmarks in Grade 1 1 . 

100 t 



On Track Off Track Far Off Track 

Preparation level in grade 8 
□ Mathematics □ Reading □ Science 


Figure 1. Percent of Kentucky students meeting the ACT College Readiness Benchmarks in 
grade 11, given their eighth grade ACT Explore performance 


21 Results for English are not included in this chart because, as noted earlier, few students scored at the Far Off 
Track level of 8 or below on the ACT Explore English exam. 
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How Did the Percentage of Far Off Track 8 th Graders Reaching Benchmarks by 11 th Grade 
Vary Across Student Demographic Groups? 

Figure 2 shows how the percentages of Far Off Track eighth grade students reaching 
College Readiness Benchmarks by eleventh grade varied between low-income and non-low- 
income students. Figure 3 provides the same information by student ethnic group, and Figure 4 
provides this information for English language learner and special education students 2 These 
charts show that Far Off Track students from at-risk groups such as low-income students, 
minority students, English language learners, and special education students reached the 
Benchmarks at lower rates than did their less at-risk counterparts. This is a matter of concern 
given that students from these groups were more likely to be Far Off Track in the first place 
(Tables 10 and 1 1). 
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Figure 2. Percent of Far Off Track eighth grade students meeting College Readiness Benchmarks 
on the ACT in grade 1 1, by student income group. 


23 The large amount of white space in these charts is deliberate, intended to emphasize the low rate of catching up of 
Far Off Track students from all demographic groups. 
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African American Hispanic Other 

Student population 

□ Mathematics D Reading □ Science 


Figure 3. Percent of Far Off Track eighth grade students meeting College Readiness Benchmarks 
on the ACT in grade 1 1, by student ethnic group. 



Figure 4. Percent of Far Off Track eighth grade students meeting College Readiness Benchmarks 
on the ACT in grade 1 1, for English language learners and special education students. 
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How Much Growth Towards College Readiness Benchmarks Did Far Off Track Students 
Achieve in High School? 

To address this question, we disaggregated students into the z-score growth categories 
described in Table 9. We examined how many Far Off Track students either reached or moved 
up close to the Benchmark, as represented by the top two z-score growth categories. These 
students are shown in the first and second bar segments in Figures 5-8 (on pages 24-26). (The 
“Reached Benchmark” category in these charts shows the same statistics as in Figures 1-4.) For 
example, the overall percentage of Far Off Track students in the top two z-score growth 
categories (reaching the Benchmark or moving to no more than a half standard deviation below 
it) was 3% in mathematics, 17% in reading, and 9% in science (Figure 5). 24 For low-income 
students, the corresponding totals were 2% in mathematics, 13% in reading, and 6% in science 
(Figure 6). 

It is also useful to look at the percentage of Far Off Track students who remained Far Off 
Track, represented by the lowest growth category in Table 9 and the last bar segment in Figures 
5-8. For low-income students, these percentages were 86% in mathematics, 66% in reading, and 
73% in science (Figure 6). African American students, English language learners, and special 
education students were the most at-risk groups based on the percentage of students staying Far 
Off Track in high school: 87, 70, and 76 percent of African American students remained Far Off 
Track in mathematics, reading, and science, respectively (Figure 7), while the corresponding 


24 Rounding may cause totals in the charts to differ from 100% and subtotals to differ from those reported in the text. 
For example, in science in Figure 9, 3.32% of students in the first category and 5.34% in the second category add up 
to 8.66% in the two categories combined. 
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statistics for English language learners were 87, 75, and 78 percent, and for special education 
students, 92, 77, and 80 percent (Figure 8). 25 
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Figure 5. Percent of Kentucky Far Off Track students changing academic preparation levels in 
grades 8-1 1, by subject. 


25 As discussed in the methodology section, score metrics based on z-scores cannot be used to examine Matthew 
effects, as dividing by the standard deviation removes the effect of increasing variance in test scores over time. 
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Figure 6. Percent of Kentucky Far Off Track students changing academic preparation levels in 
grades 8-1 1, by subject and income. 
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Figure 7. Percent of Kentucky Far Off Track students changing academic preparation levels in 
grades 8-1 1, by subject and ethnicity. 
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Figure 8. Percent of Kentucky Far Off Track English language learners and special education 
students changing academic preparation levels in grades 8-1 1, by subject. 


Closing Academic Preparation Gaps between Grades 4 and 8 
What Percentage of Students Were Far Off Track in 4 th Grade? 

Table 12 shows the percentage of fourth grade KCCT test-takers in Kentucky who were 
Far Off Track in the 2006-07 and 2007-08 school years, the starting years for the students in the 
two Grades 4-8 cohorts in this study. 39, 42, and 44 percent of students were Far Off Track in 
mathematics, reading, and science, respectively (top row of Table 12). These percentages were 
higher for students in at-risk student groups, as can be seen from the remaining rows. Students in 
the longitudinal cohorts (Table 13) were Far Off Track at the same or lower rates. However, the 
difference between the performance of ah tested students and the subset of students in 
longitudinal cohorts was less in grades 4-8 than in high school, as shown by comparing the 
differences between Tables 12 and 13 with those between Tables 10 and 11. This could result 
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from greater attrition of worse prepared students from the high school cohorts. For example, the 
74,573 students in the “All Students” group in Table 13 represent 86% of the 86,936 fourth grade 
students in Table 12, whereas the 95,342 students in the corresponding group in Table 11 
represent 74% of the 128,51 1 eighth grade students in Table 10. In addition, cohort attrition may 
be more closely related to academic performance in high school (e.g., students drop out, are 
retained in grade) than in the middle grades. 

Table 12 

Percentage of Kentucky Tested 4 th Grade Students Who Were Far Off Track from 2006-07 and 
2007-08 School Years 


Group 

# 

Group 

Number of 
students 

Percentage of 4th graders who were 
Far Off Track 

Mathematics Reading Science 

1 

All students 

86,936 

39 

42 

44 

2 

Low-income 

45,582 

51 

52 

55 

3 

Non-low-income 

41,354 

27 

30 

32 

4 

African American 

9,086 

61 

63 

69 

5 

Hispanic 

2,304 

49 

50 

58 

6 

Other 

75,546 

37 

39 

40 

7 

English language learners 

1,502 

57 

61 

69 

8 

Special education 

11,640 

60 

59 

61 
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Table 13 


Percentage of Kentucky Tested 4 th Grade Students Who Were Far Off Track from Students in 
Longitudinal Cohorts, 2006-07 and 2007-08 School Years 


Group 

# 

Group 

Number of 
students 

Percentage of 4th graders who were 
Far Off Track 

Mathematics Reading Science 

1 

All students 

74,573 

38 

41 

43 

2 

Low-income 

38,148 

49 

51 

53 

3 

Non-low-income 

36,425 

26 

30 

31 

4 

African American 

7,397 

60 

62 

68 

5 

Hispanic 

1,715 

46 

49 

56 

6 

Other 

65,461 

35 

38 

39 

7 

English language learners 

1,059 

56 

60 

69 

8 

Special education 

9,560 

59 

58 

60 


What Percentage of Far Off Track 4 th Graders Were On Track by 8 th Grade? 

For the Kentucky longitudinal cohorts in the study, 7, 8, and 12 percent of Far Off Track 
fourth graders reached the ACT Explore College Readiness Bench marks by eighth grade in 
mathematics, reading, and science, respectively (Figure 9). Success rates for Off Track students 
were higher: 32, 36, and 37 percent in those three subjects. As was the case in high school, the 
majority of On Track fourth graders were still on track four years later. 
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Figure 9. Percentage of Kentucky students meeting the ACT College Readiness Benchmarks on 
Grade 8 ACT Explore, given their fourth grade KCCT performance. 

How Did the Percentage of Far Off Track 4 th Graders Getting On Track by 8 th Grade Vary 
Across Student Demographic Groups? 

Figure 10 shows how the percentages of Far Off Track fourth grade students getting on 
track by the eighth grade ACT Explore varied between low-income and non-low-income 
students. Figure 11 provides the same information by student ethnic group, and Figure 12 
provides this information for English language learners and special education students. These 
charts show that Far Off Track students from at-risk groups caught up at lower rates than did 
their less at-risk counterparts. Students from these groups were also more likely to be Far Off 
Track in the first place, as shown in Tables 12 and 13. 26 


26 Results for other disaggregated student groups are available in Appendix B. 
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Low -income Non -low -income 


Student population 

I Mathematics H Reading □ Science 

Figure 10. Percentage of Far Off Track fourth grade students reaching the ACT Explore 8 th 
grade Benchmarks, by student income group. 



Student population 

□ Mathematics H Reading □ Science 

Figure 11. Percentage of Far Off Track fourth grade students reaching the ACT Explore 8 th 
grade Benchmarks, by student ethnic group. 
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English language learners Special education 

Student population 

I Mathematics B Reading □ Science 

Figure 12. Percentage of Far Off Track fourth grade students reaching the ACT Explore 8 th 
grade Benchmarks, for English language learners and special education students. 

How Much Growth Towards 8 th Grade Benchmarks Did Far Off Track Students Achieve 
between Grades 4 and 8? 

Using the growth categories in Table 9, we examined the percentage of Far Off Track 
students who either reached the Benchmark or moved up into the top half of the Off Track 
category, indicating that they were getting close to the Benchmark (Figures 13-16 on pages 32- 
34). (The “reached benchmark” category in these charts reports the same statistics as in Figures 
9-12.) The overall percentage of Far Off Track students in the top two growth categories was 
18% in mathematics, 15% in reading, and 27% in science (Figure 13). For low-income students, 

97 

the corresponding totals in those three subjects were 15, 13, and 22 percent (Figure 14). 

It is also useful to look at the percentage of Far Off Track students who remained Far Off 
Track, represented by the last bar segment in Figures 13-16. For low-income students, these 

27 


See footnote 24 about rounding. 
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percentages were 49% in mathematics, 62% in reading, and 47% in science (Figure 14). African 
American and special education students were the most at-risk groups based on the percentage of 
students staying Far Off Track in grades 4-8: 58, 70, and 53 percent of African American 
students remained Far Off Track in mathematics, reading, and science, respectively (Figure 15), 
while the corresponding statistics for special education students were 64, 70, and 61 percent 
(Figure 16). 
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Figure 13. Percent of Kentucky Far Off Track students changing academic preparation levels 
between Grades 4 and 8, by subject. 
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Figure 14. Percent of Kentucky Far Off Track students changing academic preparation levels 
between Grades 4 and 8, by subject and income. 
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Figure 15. Percent of Kentucky Far Off Track students changing academic preparation levels 
between Grades 4 and 8, by subject and ethnicity. 
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Figure 16. Percent of Kentucky Far Off Track English language learners and special education 
students changing academic preparation levels between Grades 4 and 8, by subject. 


How Did Growth by Far Off Track Students in the Middle Grades Compare with Growth 
by Far Off Track Students in High School? 

Growth comparisons between grades 4-8 and 8-11 can be difficult to make because of 
differences in selection effects between the two levels. These selection effects ought to favor 
growth by students in high school cohorts, as attrition is greater in high school. Attrition is 
likely to remove a disproportionate share of less prepared and slower growing students, who are 
more likely to drop out, be retained in grade, and not take the ACT. 

A second issue complicating comparisons between grades 4-8 and 8-11 is differences in 
the content alignment of the fourth grade KCCT tests with the eighth grade ACT Explore, 
compared with the alignment of ACT Explore with the ACT. Differences in content tested 

28 The greater selectivity of the high school cohorts can be seen by comparing the size of the cohorts with the total 
number of tested students in Tables 2 and 4. Also, in grade 8 the students in longitudinal cohorts did better relative 
to all tested students (Table 1 1 vs. Table 10) than was the case in grade 4 (Tables 13 vs. Table 12). 
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reduce the correlation between the results from earlier and later tests, increasing regression 
effects (Campbell & Kenny, 1999).“ Thus, lower alignment of the grades 4 and 8 tests would 
produce more “growth” between grades 4 and 8 for the students who are farthest off track, 
compared with the growth shown by similar students between grades 8 and 1 1 on more highly 
aligned tests. This effect is likely to work in the opposite direction from selection effects, 

30 

favoring growth in the middle grades over high school. 

With these caveats, Tables 14-16 on pages 37-39 compare the percentage of students 
from each group who made it into the top two perfonnance levels by the end of the period (grade 
8 for fourth graders or grade 1 1 for eighth graders), summarizing infonnation from Figures 5-8 
and Figures 13-16. In addition, these tables provide infonnation on how far below the On Track 
level each group of students started out on average, measured in standard deviation units. This 
information is shown in the columns labeled “average distance below benchmark in grade 4 (or 
8 ).” 

In mathematics, the students in the grades 4-8 cohorts started out roughly the same 
distance behind as did students in the grades 8-11 cohorts, but in general were more successful at 
making it into the top two levels by the end of the cohort period. For example, African American 
students in the grades 4-8 cohorts started out an average of 1.87 standard deviations below On 
Track performance levels in fourth grade; their counterparts in the grades 8-11 cohorts started 
out about 1.86 standard deviations behind (Table 14). Yet 10% of African American students 
were able to transition into the top two perfonnance levels in the middle grades, versus 2% in 

29 A longer time interval between earlier and later tests would also tend to reduce the correlation between the scores. 
However, because the fourth and eleventh grade tests were given in the spring and the eighth grade test in the fall, 
the average time intervals between the two sets of tests were not very different: three years and four months for 
grades 4-8, versus three years and six months for grades 8-11. 

30 Correlations between grade 4 KCCT and grade 8 ACT Explore scores were lower than between grade 8 ACT 
Explore and grade 11 ACT scores: for the former, correlations in mathematics, reading, and science were .608, .570, 
and .518, respectively, versus .704, .708, and .636 between grades 8 and 11. 
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high school. This provides some evidence that it may be easier for students to catch up in 
mathematics in the middle grades than in high school. 

In reading, students caught up at similar rates in grades 4-8, even though they started out 
farther behind (Table 15). An ordinary least squares regression of z-score growth versus initial 
scores provides evidence that Kentucky Far Off Track students who were equally far behind in 
reading did more catching up in the middle grades than in high school. 31 In science, students 
started out farther behind in grades 4-8 but caught up at higher rates than in grades 8-11 (Table 
16), a result also supported by regression analysis. ~ However, the correlation between students’ 
fourth and eighth grade science scores was relatively low, a likely indication of low content 
alignment between the two tests. ~ In general, differences in content alignment between the 
fourth grade state test and ACT Explore, as compared with the alignment of ACT Explore with 
the ACT, are likely to be an important issue affecting comparisons in all three subjects. 
Therefore, further evidence is needed to determine whether catching students up in each subject 
is easier in the middle grades than in high school, and whether current efforts to remediate 
students in middle school are more effective than similar efforts in high school. 


31 For example, a student who started out 1.5 standard deviations below the On Track level in fourth grade reading 
was predicted to move .72 standard deviations closer to the Benchmark between fourth and eighth grade, while a 
student in a similar position in eighth grade was predicted to move only. 29 standard deviations closer to the 
Benchmark between grades 8 and 11. Similar results obtained when score change per year was used as the metric to 
allow for differences in students’ time to grow. 

32 For example, a student who started out 1.5 standard deviations below the On Track level in fourth grade science 
was predicted to move .65 standard deviations closer to the Benchmark between fourth and eighth grade, while a 
student in a similar position in eighth grade was predicted to move only .10 standard deviations closer to the 
Benchmark between grades 8 and 1 1 . 

33 In support of our hypothesis of relatively low content alignment between the fourth and eighth grade science tests, 
the correlation between students’ fourth and eighth grade science scores (.518) was lower than that between their 
fourth grade science scores and their eighth grade mathematics and reading scores (.533 and .524, respectively). 
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Table 14 

Percent of Kentucky Far Off Track Students Reaching the Top Two Mathematics Performance 
Levels: Grades 4-8 versus Grades 8-11 


Group 

Number 
of Far Off 
Track 
students 

Grades 4-8 
Average 
distance 
below 
benchmark 
in grade 4 

Percent in 
top two 
growth 
categories 

Number 
of Far Off 
Track 
students 

Grades 8-11 
Average 
distance 
below 
benchmark 
in grade 8 

Percent in 
top two 
growth 
categories 

All students 

28,387 

-1.73 

18% 

24,967 

-1.79 

3% 

Low-income 

18,780 

-1.79 

15% 

15,137 

-1.83 

2% 

Non-low-income 

9,607 

-1.63 

24% 

9,830 

-1.73 

4% 

African American 

4,404 

-1.87 

10% 

3,728 

-1.86 

2% 

Hispanic 

794 

-1.77 

16% 

508 

-1.79 

3% 

Other 

23,189 

-1.71 

20% 

20,731 

-1.78 

3% 

English language 
learners 

591 

-1.84 

15% 

293 

-1.93 

3% 

Special education 

5,643 

-2.01 

11% 

5,880 

-2.10 

2% 
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Table 15 

Percent of Kentucky Far Off Track Students Reaching the Top Two Reading Performance 
Levels: Grades 4-8 versus Grades 8-11 


Group 

Number 
of Far Off 
Track 
students 

Grades 4-8 
Average 
distance 
below 
benchmark 
in grade 4 

Percent in 
top two 
growth 
categories 

Number of 
Far Off 
Track 
students 

Grades 8-11 

Average 
distance 
below 
benchmark 
in grade 8 

Percent in 
top two 
growth 
categories 

All students 

30,281 

-1.80 

15% 

30,995 

-1.34 

17% 

Low-income 

19,444 

-1.86 

13% 

17,576 

-1.36 

13% 

Non-low-income 

10,837 

-1.68 

20% 

13,419 

-1.31 

23% 

African American 

4,565 

-1.99 

8% 

4,162 

-1.38 

10% 

Hispanic 

844 

-1.81 

12% 

592 

-1.34 

13% 

Other 

24,872 

-1.76 

17% 

26,241 

-1.33 

19% 

English language 
learners 

640 

-1.86 

12% 

345 

-1.38 

10% 

Special education 

5,528 

-1.96 

9% 

5,554 

-1.41 

10% 
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Table 16 

Percent of Kentucky Far Off Track Students Reaching the Top Two Science Performance Levels: 
Grades 4-8 versus Grades 8-11 


Group 

Number 
of Far Off 
Track 
students 

Grades 4-8 
Average 
distance 
below 
benchmark 
in grade 4 

Percent in 
top two 
growth 
categories 

Number of 
Far Off 
Track 
students 

Grades 8-11 

Average 
distance 
below 
benchmark 
in grade 8 

Percent in 
top two 
growth 
categories 

All students 

31,716 

-1.72 

27% 

22,602 

-1.59 

9% 

Low-income 

20,386 

-1.80 

22% 

13,179 

-1.62 

6% 

Non-low-income 

11,330 

-1.59 

36% 

9,423 

-1.56 

12% 

African American 

5,043 

-1.96 

16% 

3,015 

-1.63 

4% 

Hispanic 

958 

-1.80 

24% 

428 

-1.55 

5% 

Other 

25,715 

-1.67 

29% 

19,159 

-1.58 

9% 

English language 
learners 

729 

-1.88 

24% 

239 

-1.62 

5% 

Special education 

5,738 

-1.97 

14% 

4,986 

-1.74 

5% 
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Conclusion 

The results in this study on the difficulty of catching up Far Off Track students are 
consistent with previous research findings (Sawyer, 2008; ACT, 2008, 2012a; Dougherty, 2010; 
Dougherty & Fleming, 2012; Sawyer & Gibson, 2012). This study extends those findings to 
demographic subgroups such as low-income students, African Americans, Hispanics, English 
language learners, and special education students. When more states provide the necessary data, 
research on students catching up by demographic subgroups in those states will be possible. 34 

These results support a general finding that it is difficult for students who are far behind 
to get on track in middle or high school. While overestimating the difficulty of catching up might 
encourage educators and policymakers to give up on students, underestimating the difficulty 
might lead educators to choose strategies and interventions that are too little and too late. For 
their part, policymakers who think that catching students up is easier than it actually is may 
reduce funding for educational programs. They may also hold schools to accountability targets 
that are not attainable over the period in question, creating strong incentives for leaders at 
various levels in the system to seek to artificially inflate test scores. 

The high percentage of students who are below college and career readiness achievement 
targets at all grade levels — and the difficulty of catching them up — should also lead educators 
and policymakers to focus on the importance of early learning and to emphasize prevention over 
remediation (ACT, 2012a). These prevention strategies may include: changing the regular 
academic program to give every student access to a content- and vocabulary-rich curriculum 
beginning in the early years (Willingham, 2009; Common Core State Standards Initiative, 2010; 

34 For example, the Arkansas Department of Education provided similar data, making possible a research report on 
students catching up in that state (Dougherty, Hiserote, & Shaw, 2014, in press). Dougherty & Fleming (2012) 
examined the percentages of Far Off Track students who caught up in four multi-state student cohorts in grades 8-12 
and two statewide Arkansas cohorts in grades 4-8. 
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ACT, 2012c; Dougherty, 2013); strengthening the early reading and mathematics program in 
preschool through third grade; and implementing programs and strategies that improve students’ 
attendance and academic behaviors (Diamond, Barnett, Thomas, & Munro, 2007a, 2007b; 
Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011; Sawyer & Gibson, 2012). Efforts to 
close academic preparation gaps should begin as early as possible, be more intensive, and take as 
long as necessary. Even if starting earlier does not reduce the amount of time it takes for students 
to catch up, starting earlier gives them more time to catch up. 

In addition, a database might be developed to learn more about how effective various 
programs and interventions are at helping students catch up — from how far behind and over what 
length of time (Dougherty, 2010). Key components of the database would include information on 
how far behind the students are at different points in time and on the nature, length, and intensity 
of the interventions they receive. From this, it might be possible to identify combinations of 
curriculum, interventions, and time requirements that are sufficient to enable most off-track 
students to succeed. 

These findings should also affect the requirements that accountability systems place on 
schools. For example, reasonable growth goals might be set based on student performance in 
more successful schools (ACT, 2009, 2012d), and goals for percentages of students reaching 
college and career readiness should take into account the students’ starting points and the number 
of years the school has available to catch them up, as is done in value-added models. In general, 
policy and practice should be informed by data on the success of real students in actual schools. 
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Narrowing of Scale Score Gaps in Grades 8-11 

Because the ACT Explore and ACT tests are scored on a common scale, growth between 
those two tests can be measured in scale score points. Accordingly, we disaggregated students in 
the longitudinal grades 8-11 cohorts into the scale score categories described in Table 8. Figure 
A1 on page 50 shows the percent of Far Off Track students in the three Kentucky high school 
cohorts falling into each of these categories. Figures A2, A3, and A4 on pages 50-51 provide the 
same information for student demographic groups based on income, ethnicity, and English 
language learner and special education status. (The “Reached Benchmark” category in these 
charts shows the same statistics as in Figures 1-4.) 

As can be seen from these charts, the majority of Far Off Track students from all student 
groups did not narrow their ACT Explore scale score gaps on the ACT. For example, scale score 
gaps on the ACT remained the same or widened for 71% of students in mathematics, 76% in 
reading, and 74% in science (Figure Al). The percentage of Far Off Track students in the first 
two scale score growth categories (reaching the Benchmark or closing their scale score gaps by 
half or more) was 5% in mathematics, 13% in reading, and 1 1% in science (Figure Al). For low- 
income students, the corresponding totals were 4% in mathematics, 10% in reading, and 8% in 
science (Figure A2). ~ These results are broadly similar to those from the z-score analysis. All of 
this could indicate the presence of Matthew effects and the lack of sufficiently intensive and 
comprehensive interventions for these students in high school. 


35 Totals on the charts may differ from 100%, and subtotals on the charts from those reported in the text, due to 
rounding. For example, in mathematics in Figure Al, 1.4% of students in the first category and just under 3.5% of 
students in the second category add up to 4.9% in the two categories combined, rounding to 5%, even though the 
first two percentages rounded are 1% and 3%. 
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Appendix B 

Results by Income, Ethnicity, and Gender 
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Table B1 

Percentage of All Kentucky ACT Explore Tested Students Who Were Far Off Track in 8 th 
Grade, 2006-07, 2007-08, and 2008-09 School Years 







Percentage of 8th graders who were Far 







Off Track 






Number of 




Group 36 

Income 

Ethnicity 

Gender 

students 

Mathematics 

Reading 

Science 

9 

Low- 

income 

African 

American 

Male 

Female 

4,568 

59.7% 

68.8% 

53.7% 

10 

4,561 

54.6% 

54.6% 

41.2% 

11 

Low- 

income 

Hispanic 

Male 

Female 

963 

47.4% 

56.4% 

42.8% 

12 


853 

46.8% 

48.9% 

35.4% 

13 

Low- 

Other 

Male 

26,323 

44.2% 

53.0% 

42.9% 

14 

income 


Female 

25,383 

38.2% 

40.4% 

31.5% 

15 

Non- 

low- 

African 

American 

Male 

Female 

1,596 

40.4% 

50.8% 

37.4% 

16 

income 

1,566 

34.5% 

35.0% 

25.9% 

17 

Non- 


Male 

271 

25.8% 

34.7% 

25.8% 


low- 

Hispanic 






18 

income 


Female 

273 

26.7% 

27.8% 

20.9% 

19 

Non- 


Male 

31,954 

22.1% 

31.6% 

23.3% 


low- 

Other 






20 

income 


Female 

30,200 

18.1% 

21.4% 

15.5% 


36 The numbering of student demographic groups is based on the list on pages 7 and 8. 
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Table B2 

Percentage of Kentucky Students in Longitudinal Cohorts Who Were Far Off Track in 8 th Grade 
from 2006-07, 2007-08, and 2008-09 School Years 


Percentage of 8th graders who were Far 
Off Track 

Number of 


Group 

Income 

Ethnicity 

Gender 

students 

Mathematics 

Reading 

Science 

9 

Low- 

income 

African 

American 

Male 

Female 

2,682 

53.4% 

63.8% 

46.6% 

10 

3,079 

48.5% 

49.2% 

35.4% 

11 

Low- 

income 

Hispanic 

Male 

561 

38.1% 

47.1% 

35.8% 

12 


Female 

526 

39.2% 

43.0% 

28.3% 

13 

Low- 

Other 

Male 

16,628 

36.3% 

45.9% 

35.5% 

14 



Female 

17,527 

32.9% 

35.5% 

26.2% 

15 

Non- 

low- 

African 

American 

Male 

Female 

1,153 

36.1% 

47.1% 

33.6% 

16 

income 

1,246 

31.0% 

31.5% 

23.1% 

17 

Non- 


Male 

177 

20.9% 

28.8% 

21.5% 


low- 

Hispanic 






18 

income 


Female 

193 

26.4% 

26.4% 

20.7% 

19 

Non- 


Male 

25,878 

18.5% 

28.3% 

20.0% 


low- 

Other 






20 

income 


Female 

25,692 

16.2% 

19.7% 

13.6% 


Percent of Kentucky Far Off Track 8 th Grade Students Meeting ACT College Readiness Benchmarks in Grade 1 1 
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Table BIO 

Percentage of All Tested Kentucky Students Who Were Far Off Track in 4 th Grade from Students 
Taking Kentucky Benchmark Exams, 2006-07 and 2007-08 School Years 







Percentage of 4th graders who were Far Off 







Track 






Number 








of 




Group 

Income 

Ethnicity 

Gender 

students 

Mathematics 

Reading 

Science 

9 

Low- 

African 

America 

Male 

3,613 

65.9% 

71.8% 

72.9% 

10 


n 

Female 

3,600 

64.0% 

61.2% 

73.6% 

11 

Low- 

income 

Hispanic 

Male 

974 

51.3% 

55.6% 

58.3% 

12 


Female 

902 

52.8% 

49.7% 

65.1% 

13 

Low- 

Other 

Male 

18,621 

47.0% 

53.4% 

49.5% 

14 



Female 

17,872 

48.4% 

44.7% 

51.8% 

15 

Non- 

African 

Male 

952 

44.8% 

52.8% 

54.3% 


low- 

America 






16 

income 

n 

Female 

921 

44.5% 

42.2% 

54.0% 

17 

Non- 


Male 

218 

32.4% 

42.9% 

41.1% 


low- 

Hispanic 






18 

income 


Female 

210 

36.7% 

31.4% 

40.5% 

19 

Non- 


Male 

20,033 

26.5% 

34.9% 

30.2% 


low- 

Other 






20 

income 


Female 

19,020 

25.9% 

23.7% 

31.2% 
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Table Bll 

Percentage of Kentucky Students in Longitudinal Cohorts Who Were Far Off Track in 4' h Grade from 
Students Taking Kentucky Benchmark Exams, 2006-07 and 2007-08 School Years 


Percentage of 4th graders who were Far Off 
Track 

Number of 


Group 

Income Ethnicity 

Gender 

students 

Mathematics 

Reading 

Science 

9 

Low- African 
income American 

Male 

Female 

2,928 

64.6% 

71.3% 

71.7% 

10 


2,890 

63.0% 

60.2% 

72.4% 

11 

Low- TT . 

Flispamc 

income 

Male 

742 

48.8% 

54.4% 

56.1% 

12 

Female 

650 

49.4% 

48.6% 

63.2% 

13 

. L° W Other 

income 

Male 

15,587 

45.6% 

52.3% 

48.4% 

14 

Female 

15,351 

47.4% 

44.0% 

51.0% 

15 

Non-low- African 

Male 

798 

44.5% 

52.6% 

54.8% 


income American 

Female 





16 


781 

43.1% 

40.5% 

53.0% 

17 

Non-low- TT . 

Hispanic 

income 

Male 

180 

33.3% 

43.3% 

40.0% 

18 

Female 

143 

35.7% 

32.2% 

41.3% 

19 

Non-low- _ . 

Male 

17,494 

25.7% 

34.3% 

29.4% 


Other 






20 


Female 

17,029 

25.3% 

23.4% 

30.5% 


Percentage of Kentucky Far off Track 4 th Grade Students Meeting College Readiness Benchmarks on 8 th Grade ACT Explore 
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